Comparison of the Goodness-of-Fit Tests: the Pearson Chi-square and Kolmogorov-Smirnov Tests

A test for goodness of fit usually involves examining a random sample from some unknown distribution in order to test the null hypothesis that the unknown distribution function is in fact a known, specified function. The Chi-square test can be applied to any univariate distribution for which you can calculate the cumulative distribution function. The Chi-square test does not have good properties (power and type I error rate) for small sample sizes. The Kolmogorov-Smirnov goodness-of-fit test (KS test) is alternative to the Chi-square test. This paper summarizes the results of a Monte-Carlo investigation of the Type I error rate and power of both tests. In every case examined where a 0.05 significance level was targeted, the simulated type I error rate of the KS test is smaller than that of the Chi-square test. The power of the Chi-square test is smaller than the KS test except when the distribution in the null hypothesis has a given mean (regardless of the spread of the distribution). Therefore, the KS test is more valid than the Chi-square test in general when a 0.05 significance level was targeted.